How Computers Discover How Computers Discover (A mini-review of algorithmic meta-discovery)

In traditional exerimental sciences one forms hypotheses explaining an empirical sample of some natural phenomenon. The hypothesis discovery process can in certain cases be automated using machine learning algorithms. I discuss some consequences of viewing scientific discovery, primarily computer based, itself as a natural phenomenon, whose empirical sample may be analyzed or hypothesized about. This viewpoint frames some recent findings about the statistical properties of computationally intensive discovery processes (phase transitions in complexity of learning, heavy-tailed runtime distributions of hypothesis search), empirical assessments of the Occam’s razor principle, as well as the issue of meta learning (such as ‘discovering how to discover best’). Here I review these points and conclude with a few speculative thoughts.